Linearized Alternating Direction Method of Multipliers via Positive-Indefinite Proximal Regularization for Convex Programming
نویسندگان
چکیده
The alternating direction method of multipliers (ADMM) is being widely used for various convex minimization models with separable structures arising in a variety of areas. In the literature, the proximal version of ADMM which allows ADMM’s subproblems to be proximally regularized has been well studied. Particularly the linearized version of ADMM can be yielded when the proximal terms are appropriately chosen; and for some applications it could alleviate an ADMM subproblem as easy as estimating the proximity operator of a function in the objective. This feature is significant in easing the numerical implementation and it makes the linearized version of ADMM very popular in a broad spectrum of application domains. To ensure the convergence of the proximal version of ADMM, however, existing results conventionally need to guarantee the positive definiteness of the corresponding proximal matrix. For some cases, this essentially results in small step sizes (or, over-regularization effectiveness) for the subproblems and thus inevitably decelerates the overall convergence speed of the linearized version of ADMM. In this paper, we investigate the possibility of relaxing the positive definiteness requirement of the proximal version of ADMM and show affirmatively that it is not necessary to ensure the positive definiteness of the proximal matrix. A new linearized ADMM with larger step sizes is thus proposed via choosing a positive-indefinite proximal regularization term. The global convergence of the new linearized ADMM is proved; and its worst-case convergence rate measured by the iteration complexity is also established. Since the ADMM can be regarded as a splitting version of the augmented Lagrangian method (ALM), a byproduct of our analysis is a new linearized version of ALM generated by choosing a positive-indefinite proximal regularization term for its subproblems.
منابع مشابه
The symmetric ADMM with positive-indefinite proximal regularization and its application
Due to update the Lagrangian multiplier twice at each iteration, the symmetric alternating direction method of multipliers (S-ADMM) often performs better than other ADMM-type methods. In practice, some proximal terms with positive definite proximal matrices are often added to its subproblems, and it is commonly known that large proximal parameter of the proximal term often results in “too-small...
متن کاملThe convergence rate of the proximal alternating direction method of multipliers with indefinite proximal regularization
The proximal alternating direction method of multipliers (P-ADMM) is an efficient first-order method for solving the separable convex minimization problems. Recently, He et al. have further studied the P-ADMM and relaxed the proximal regularization matrix of its second subproblem to be indefinite. This is especially significant in practical applications since the indefinite proximal matrix can ...
متن کاملAn inexact alternating direction method with SQP regularization for the structured variational inequalities
In this paper, we propose an inexact alternating direction method with square quadratic proximal (SQP) regularization for the structured variational inequalities. The predictor is obtained via solving SQP system approximately under significantly relaxed accuracy criterion and the new iterate is computed directly by an explicit formula derived from the original SQP method. Under appropriat...
متن کاملLinearized Alternating Direction Method of Multipliers for Constrained Nonconvex Regularized Optimization
In this paper, we consider a wide class of constrained nonconvex regularized minimization problems, where the constraints are linearly constraints. It was reported in the literature that nonconvex regularization usually yields a solution with more desirable sparse structural properties beyond convex ones. However, it is not easy to obtain the proximal mapping associated with nonconvex regulariz...
متن کاملThe symmetric ADMM with indefinite proximal regularization and its application
Due to updating the Lagrangian multiplier twice at each iteration, the symmetric alternating direction method of multipliers (S-ADMM) often performs better than other ADMM-type methods. In practical applications, some proximal terms with positive definite proximal matrices are often added to its subproblems, and it is commonly known that large proximal parameter of the proximal term often resul...
متن کامل